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Abstract 

We extend our earlier work in [TZ1], where an analytic approach 
to the Guillemin-Sternberg conjecture [GS] was developed, to cases 
where the Spin c -complex under consideration is allowed to be fur- 
ther twisted by certain natural exterior power bundles. The main 
result is a weighted quantization formula in the presence of commut- 
ing Hamiltonian actions. The corresponding Morse type inequalities 
in holomorphic situations are also established. 

§0. Introduction 

In a previous paper [TZ1], we have developed a direct analytic approach 
to, as well as certain extensions of, the Guillemin-Sternberg geometric quan- 
tization conjecture [GS], which has been proved in various generalities in 
([DGMW, G, GS, JK, Ml, M2, VI, V2]). In this paper, we generalize the re- 
sults in [TZ1] to cases where the Spin c -complex under consideration is allowed 
to be further twisted by certain natural exterior power bundles. The main 
result is a weighted quantization formula for these twisted Spin c -complexes 
in the presence of commuting Hamiltonian actions. We also establish the 
corresponding Morse type inequalities in the holomorphic situation. 
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To begin with, let (M, uj) be a closed symplectic manifold admitting a 
Hamiltonian action of a compact connected Lie group G with Lie algebra g. 
Let J be an almost complex structure on TM so that g™(u,v) = u(u, Jv) 
defines a Riemannian metric on TM. After an integration over G if nec- 
essary, we can and we will assume that G preserves J and g™ . Let E 
be a G-equivariant Hermitian vector bundle with a G-equivariant Hermitian 
connection V s . 

With these data in hand, for any integer p > 0, one can construct canon- 
ically a formally self-adjoint twisted Spin c -Dirac operator acting on smooth 
sections of the twisted Spin c -vector bundles: 

D A»°(T*M)®£ . QP.even^ ^ ^<° dd (M, £). (0.1) 

It gives rise to the finite dimensional virtual vector space 

Q(M, A^°(T*M) ® £) = ker jD ^'°( r ^)^ s _ coke r ^a- ^^)^^. (0 . 2 ) 

Since G preserves everything, one sees easily that Q(M, A P '°(T* M)®E) is 
a virtual representation of G. Denote by Q(M, A p '°(T*M)<g>E) G its G-trivial 
component. 

Let g (and thus its dual g* also) be equipped with an Ad G-invariant 
metric. Let hi, 1 < i < dimG, be an orthonormal base of g*. Let Vi, 
1 < i < dimG, be the dual base of {/ii}i<i<dimG- 

Let : M — > g* be the moment map of the G action on M. Then it can 
be written as 

dimG 

A* = Vi^h (0-3) 
i=i 

with each /ij a real function on M. 
Now for each V G g, setQ 

rf = L£ - V^, (0.4) 
where L¥ f denotes the infinitesimal action of V on E. 



1 When there is no confusion, in this paper we will use the same notation V for the 
Killing vector field it generates on M . 
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Definition 0.1. We say E is ^-positive if the inequality 



dim G 



i J2 ^K( x ) > 



i 



(0.5) 



i=i 



holds at every critical point x e M\/i 1 (0) o/ |/i| 2 7 i/ie norm square of the 
moment map. 

As a typical example, the G-equivariant prequantum line bundle over 
(M,u) verifying the Kostant formula ([Ko], cf. [TZ1, (1.13)]), when it exists, 
is //-positive. 

To state the main results of this paper, we now assume that G g* is 
a regular value of ji and, for simplicity, that G acts freely on /i _1 (0). Then 
one can construct the Marsden-Weinstein reduction (Mq,ouq), which is a 
smooth symplectic manifold with M G = fi~ 1 (0)/G and the symplectic form 
ujg descended from uj. The almost complex structure J also descends to an 
almost complex structure on TM G . Furthermore, E descends to a Hermitian 
vector bundle E G over Mq with an induced Hermitian connection. Thus one 
can make the same construction of the twisted Spin c -Dirac operators as well 
as the associated virtual vector spaces Q(M G , A P '°(T*M G ) <g> E G ). 

For any integer k, s > 0, let be the binomial coefficient given by 



The main result of this paper, which is a generalization of [TZ1, Theorem 
4.1] in the abelian group action case, can be stated as follows. 

Theorem 0.2. If G is abelian and E is ^-positive, then the following 
identity holds for any integer p > 0, 



dimQ(M, A P '°(T*M) ® E) G = ^ C P ^ G ■ dimQ(M G , A k '°(T*M G ) <g> E G ). 



When p = and E is the prequantum line bundle (when it exists) over 
(M,u), (0.7) is the abelian version of the Guillemin- Sternberg conjecture 
[GS] proved by Guillemin [G] in a special case and by Meinrenken [Ml] and 




s(s-l)---(s-k + l) 
k\ 



(0.6) 



p 



k=0 



(0.7) 



3 



Vergne [VI, 2] in general (see also [DGMW] and [JK]). Thus in some sense 
one may view (0.7) as a kind of weighted quantization formula. 

We will use the analytic approach developed in [TZ1] to prove Theorem 
0.2. However, it should be pointed out that Theorem 0.2 is not a consequence 
of the result in [TZ1, Theorem 4.1], which itself is a generalization of the 
Guillemin-Sternberg conjecture [GS]. In particular, the strict inequality in 
(0.5) can not be relaxed to include the equality as in [TZ1, Theorem 4.2], 
even when /i _1 (0) ^ 0. Furthermore, the abelian condition on G is essential 
to both the results as well as their proofs. A notable feature here is that we 
deal with the general case, where G may possibly be of higher rank, directly. 
That is, we do not first prove the result for the G = S 1 case and then use 
the 'reduction in stage' procedure to get the full result. 

Now as in [TZ1, Theorem 0.4 and 4.8], we consider a holomorphic re- 
finement of Theorem 0.2. That is, we assume that (M,u,J) is Kahler, G 
acts on M holomorphically and E is an equivariant holomorphic Hermitian 
vector bundle over M with the G action on E being holomorphic. If for any 
integers p, q > 0, denote by 

h p ' q (E) G = dim H°' q (M, A P '°(T*M) ® E) G , 

h p ' q {E G ) = dim H°' q (M G , A P '°{T*M G ) <g> E G ) (0.8) 

the corresponding (G-invariant) twisted Hodge numbers, then we can state 
our refinement of (0.7) as follows. 

Theorem 0.3. If G is abelian and E is ^-positive, then the following 
inequality holds for any integers p, q > 0, 

h p ' q (Ef - h p ' q ~\Ef + ■■■ + (-l) q h p '°(Ef 
< E CLfc (h k > q {E G ) - h k > q -\E G ) + ■■■ + (-l) q h k \E G )) . (0.9) 

k=0 

In particular, when q = 0, one gets the following inequality for dimensions 
of spaces of holomorphic sections, 

h p '%Ef<±C p r m k G .h^(E G ). (0.10) 

fe=0 
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Again, Theorem 0.3 is not a consequence of [TZ1, Theorem 4.8]. 

This paper is organized as follows. In Section 1, we construct the twisted 
Spin c -Dirac operators appearing in the context and introduce the correspond- 
ing deformations under Hamiltonian actions as in [TZ1]. We also prove a 
Bochner type formula for the deformed operators. In Section 2, we extend 
the methods in [TZ1], which goes back to [BL], to prove Theorems 0.2 and 
0.3. The final Section 3 contains some immediate applications as well as 
further extensions of the above main results. 



§1. Deformations of twisted Spin c -Dirac operators and a Bochner 
type formula 

Following [TZ1], we construct in this section the twisted Spin c -Dirac op- 
erators and their deformations to be used in our proof of Theorems 0.2 and 
0.3. An important Bochner type formula for the deformed operators will be 
proved. 

This section is organized as follows. In a), we construct the above men- 
tioned Dirac operators. In b), following [TZ1], in the situations of Hamilto- 
nian actions we introduce the deformations of the Dirac operators constructed 
in a). In c), we prove a Bochner type formula for the deformed operators. 

a). Twisted Spin c -Dirac operators on symplectic manifolds 

Let (M, a;) be a closed symplectic manifold. Let J be an almost complex 
structure on TM such that 



defines a Riemannian metric on TM. Let TMq = TM <g> C denote the 
complexification of the tangent bundle TM. Then one has the canonical 
(orthogonal) splittings 



g (v, w) = u(v, Jw) 



(1.1) 



TM C = T ilfi) M (ST^M, 



dim c M 



A*'*(T*M) = A iJ (T*M), 



(1.2) 



i,j=0 
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where 

T (1 - 0) M = {ze TM C ; Jz = V^Tz}, 

T^ l) M = {ze TM C ; Jz = -V^lz}, 

A i,j (T*M) = A\T^>M) <g> A^T^M), (1.3) 

and dime M is the complex dimension of M. 

For any X G TM whose complexification has the decomposition X = 
X 1 +X 2 e TWA! © TWA!, let X* e T^>M (resp. X* e T^'°>M) be 
the metric dual of Xi (resp. X 2 ). Set as in [BL, Sect. 5] that 

c(X) = V2X{A -V2i X2 . (1.4) 

Then c(X) defines the canonical Clifford action of X on A°'*(T*M). In 
particular, for any X, Y E TM, one has 

c{X)c{Y) + c{Y)c{X) = -2g™(X, Y). (1.5) 

Let V™ be the Levi-Civita connection of g™ . Then V™ together with 
the almost complex structure J induce via projection a canonical Hermitian 
connection V T<1 '° >M on T^'^M. This in turn induces canonically, for any 
integer p > 0, a Hermitian connection V AP '° (T * M) on A P '°(T*M). On the other 
hand, as was shown in [TZ1], V™ lifts canonically to a Hermitian connection 
v a«.*(t*m) Qn A o,*( T * M )_ Let V Af.*(T*M) be the Hermitian connection on 

A P '*(T*M) obtained from the tensor product of V AP '° (T * M) and V A °'* {T * M) . 

Now let E be a Hermitian vector bundle over M with a Hermitian con- 
nection V E . Let v AP '* (T * M) ® £ be the tensor product connection of V AP '* (T * M) 
and V E on A P '*(T*M) <g> E. 

Denote by Q P '*(M, E) the set of smooth sections of A P '*(T*M) <g> E. 

Let ei, . . . , edimM be an orthonormal base of TM. 

Following [TZ1], we now introduce the twisted Spin c -Dirac operator we 
are interested in. 

Definition 1.1. The twisted Spirf-Dirac operator d^ v,0 { t * m )® e j s defined 

by 

dimM 

d v°(t-m)®e = ^ c(e .) V a--(t-jio®b . hp.*(m, £■) -> 1F-*(M, £). (1.6) 
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Clearly, D^ pfi ( T * M )® E j s a formally self-adjoint first order elliptic dif- 
ferential operator. Let D ^°( T * M )® E be the restriction of D^ T * M ^ E on 
QP' even (M,E). Set 

Q(M, A»°(T*M) ® E) = ker <" ° (T * M)0£ - coker £>f (1.7) 

b). Hamiltonian actions and deformations of Dirac operators 

Now suppose that (M, a;) admits a Hamiltonian action of a compact con- 
nected Lie group G with Lie algebra g. Let /x : M — > g* denote the corre- 
sponding moment map. After an integration over G if necessary, we assume 
that G preserves J and thus also g™ . We also assume that the G action on 
M lifts to a G action on E preserving the Hermitian metric as well as the 
Hermitian connection V E on E. 

Let g (and thus g* also) be equipped with an Ad G-invariant metric. Let 
7i = |/i| 2 be the norm square of the moment map /i. Then H is a G-invariant 
function on M. In particular, its Hamiltonian vector field, denoted by X n , 
is G-invariant. The following formula for X n is clear, 

x n = -j(<my. (1.8) 

Let hi, ... , h dimG be an orthonormal base of g*. Then \i has the expression 

dimG 

A« = Yl V*hi, (1.9) 
i=i 

where each /jj is a real valued function on M. Denote by Vi the Killing vector 
field on M induced by the dual of hi. One easily verifies that (cf. [TZ1, Sect, 
lb)]) 

j(dfay = (i.io) 

and 

dim G dim G 

X H = -2J Vi{dmY = 2 E VM- (1-11) 

1=1 1=1 

We are now ready to introduce the crucial deformation following [TZ1]. 
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Definition 1.2. For any TgR, let ^ M ^® E he the operator defined 

by 



D ^(T*M)®E = D A»-°(T*M)®E + V_J± C ( X H) . W»*(M,E) - W>*(M,E). 

(1.12) 

Clearly, Z)^ P ^ T M )® £ ' i s a formally self-adjoint first order elliptic differ- 
ential operator. Also, since G preserves everything and X n is G-invariant, 
one sees that ^ M ^® E i s G-equi variant. If we denote by D^ + ^ T M ^® E 
the restriction of D^ Pfi(T * M) ® E on fP' even (M, E), then 

Qt(M, A^°(T*M) ®E)= ker <; (t * m) ^ - coker D ^^ E (1.13) 

is a virtual G-representation. We use as usual a superscript G to denote its 
G-invariant component. 

Clearly, the following easy yet important identity holds for any T G R, 

dim Q T (M, A p '°(T*M) <g> £) G = dim Q(M, A P '°(T*M) <g> £) G . (1.14) 
c). A Bochner type formula for ^ T M )® E 

For any V G g, let Ly denote the infinitesimal action induced by V on 
the corresponding vector bundles. We will in general omit the superscripts 
of these bundles. Let r E be defined as in (0.4). 

For any X, Y G TM whose complexifications have the decomposition 
X = Xi + X 2 G T^M © T (0 ^M and Y = Y 1 + Y 2 G T^M © T^M 
respectively, let A(X,Y) be the endomorphism of A P '°(T*M) defined by 

A{X,Y)=X* 2 M Yl . (1.15) 

Let ei,---,edimM be an orthonormal base of TM. Then one has the 
following analogue of [TZ1, Lemma 1.5]. 

Lemma 1.3. The following identity for operators acting on Q P '*(M,E) 
holds, 

1 dimM 1 dimM 

L v = V v +r E -\ E ^)c(V^V)-i T r[v^ 0) ^]+E ^,V™F). 

(1.16) 
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Proof. By proceeding as in [TZ1, Lemma 1.5], one sees easily that one 
needs only to calculate r y P ' ^ T M \ 
Recall that V acts on TM by 



L T V M X = V T V M X - V T X M V, X e T(TM), 
from which we have 



,TM 
V 



dim M dim M 

(*) = - E (Vf^e 3 > 3 = E (V^,X)e r 
j=i j=i 



;i.i7) 



;i.i8) 



From (1.18) one gets immediately that 

dimM 



„T*M 
V 



E e j Ai vT M v- 

3=1 ' 



;i.i9) 



By (1.19), (1.15) and the fact that the almost complex structure J is 
G-invariant, one deduces easily that for any integer < p < dim c M, 



AP'°(T*M) 

"v 



dimM 



= E Mejyl M v). 



(1.20) 



1.16) then follows from (1.20) and the arguments in [TZ1, Lemma 1.5]. 



□ 



We can now state the following analogue of [TZ1, Theorem 1.6]. 
Theorem 1.4. The following Bochner type formula holds, 

dim G rp2 



i=l 4 



X 



n 



dimG 



+2v^TT ^ + ? E (v^^J^c^ + l^l 2 - Ay/=lA(JVi, Vi)) 



dimG 



i=l " i=l 

i /T-i dim M / -irp 

+ E (c(e,)c(V™X w )+4A( ej ,V™X w ))-^Tr 



;i.21) 
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Proof. As in [TZ1, (1.26) and (1.27)], one deduces from (1.12), (1.5) that 



-^rp dimM rp2 



2 



D 2 T = D 2 + — ]T c( ej )c(V ej X n ) - V^lTV x n + — X n (1.22) 

1 3=1 4 

and, by using (1.10), (1.11) and Lemma 1.4, that 

dimG dimG ^ dimM 

Vi«=2E »iL Vi - 2 E wf< + j E c(e,)c(V e3 X w ) 

i=l i=l 4 j=l 



1 



dim G / 7" dim G 



+±Tr v 3 "™** - ± E c(J^)c(^) + ^ E M 



dim M dim G 

- E ^(e„V ej X w ) + 2 E MJV t ,Vi). (1.23) 



(1.21) follows from (1.22) and (1.23). □ 



§2. Proof of the main theorems 

In this section, we apply the methods and techniques in [TZ1, Sects. 2-4], 
which are closely related to those in [BL], to prove Theorems 0.2 and 0.3. 
As in [TZ1], the key technical point is a pointwise estimate at each critical 
point x E A*V -1 (0) of Ti = |/i| 2 . 

This section is organized as follows. In a), we prove the key pointwise 
estimate mentioned above. In b), we prove Theorem 0.2 while Theorem 0.3 
will be proved in c). 

a). An estimate outside of /x _1 (0) 

Recall from Definition 0.1 that E is said to be /i-positive if (0.5) holds at 
every critical point x G M\// _1 (0) of H — |/i| 2 . 

The main result of this subsection, which is an analogue of [TZ1, Theorem 
2.1], can be stated as follows. 
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Theorem 2.1. If G is abelian and E is fi-positive, then for any open 
neighborhood U of n~ l {f)), there exist constants C > 0, b > such that for 
any T > 1 and any G -invariant section s G fl p '*(M, E) with Supp s C M\U , 
one has the following estimate of Sobolev norms, 

||< ™^ S ||^>C(|| S ||? + (T-6)|| S ||^). (2.1) 

Proof. By examining the arguments in [TZ1, Sect. 2], one sees that in 
order to prove Theorem 2.1, one needs only to prove an analogue of [TZ1, 
Lemma 2.3] in our context. 

Thus let x G M\/i _1 (0) be a critical point of T~L. Let e±, • • • , e^ixaM be 
an orthonormal base of TM near x. Let (yi, • • • , ydimhi) be the normal co- 
ordinate system with respect to {e-,}^™ M near x. Clearly, one can choose 
ei, • • • , edimM so that H has the following expression near x, 

dimM 

H{y)=H{x)+ £ %y J + 0(M 3 ), (2.2) 
i=i 

where the a/s may possibly be zero. 

We can now state our analogue of [TZ1, Lemma 2.3] as follows. 

Lemma 2.2. If G is abelian, then the following inequality holds at any 
critical point x G M\ / u~ 1 (0) ofH, 



i'dimM /_ i 

- E (c(e,)c(V™^)+4A(e,,V™^))-^-iTr 



1 dim G dim M 

+ o E (V^lci-IV^Vi) + \V l \ 2 - 4y/=lA(JVi, Vi)) >- E N- I 2 - 3 ) 

2 i=l j=l 

Proof Since x is a critical point of TC, by a result of Kirwan [K, Prop. 
3.12], x is a fixed point for the action of the subtorus generated by fi(x) ^ 0. 
Without loss of generality, we assume that hi , • • • , /idim g has been chosen so 
that the duals of hi, ■ ■ ■ , h r generate this subtorus denoted by G\. Let G2 
be the subtorus generated by the duals of h r+ i, ■ ■ ■ , hdi m G- Thus the original 
torus has the splitting 

G = Gi x G 2 . (2.4) 
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Clearly, one has that 

fjii(x) = 0, r + 1 < % < dimG. (2.5) 

Denote by F x C M the connected component containing ir of the fixed 
point set of the Gi-action. Then F x is a totally geodesic submanifold of M 
and J preserves the tangent bundle TF X . Denote k = dimi^. 

Since the GVaction commutes with the Gi-action, G2 acts on F x . 

To summarize, one has 



Lemma 2.3. a). If 1 <i < r , then Vi(x) = and ^i\f x is constant; 
b). ifr + l<i< dimG, then ^(x) = and (d^)\ Fx G T(T*F X ). 

Proof. Since G2 acts on F x , for any r + 1 < % < dimG, Vi\p x G T(TF X ). 
Thus by (1.10), (d/ii)* = JVi G Y{TF X ). The other parts of the lemma are 
clear. □ 



Without loss of generality, one may choose ei, • • • , e^imM near x so that 
ei,---,efc is an orthonormal base of TF X . Let {yj}i<j<dimM be the corre- 
sponding normal coordinates near x. Then from Lemma 2.3 one deduces 
that, near x, this orthonormal base can be further arranged so that 

r dim M 

E l/^)l 2 = IM*)I 2 + E w ? + 0(\y\ 3 ) (2.6) 

1=1 jr=fc+l 

and 

dimG k 

E k(y)l 2 = E^J + °(bl 3 )- (2.7) 

j=r+l jr' = l 

(2.6) and (2.7) together provide a splitting of H near x according to the 
splitting (2.4). 

From (1.10) and (2.6) one deduces, at x, that 

V X7™(,, V-\ - / - a i Je v for /c + 1 < j < dimM, 

v e, - 1 o for 1 < j < k. [Z -* } 

From (2.8), one deduces, at x, that 

T dimM r 

~ E E(^)c(2V™(^))+4A(e„2V™(^))) 



4 

* j=i i=i 
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1=1 



dim M / J. \ dim M 

= ~ E a i c ( e i) c ( Je i) + 2 v /Z L4(e j , J ej .) + - > - ^ | aj |, 

(2.9) 

where the last inequality follows from the obvious inequalities that 

|c(e,)c(Je,)| < 1 (2.10) 

and 

(2.11) 



2V-lA(e j ,Je j ) + - 



1 -gT* 

--2e/ AJ e i,o 



1 

£ 2' 



with e}'° G T^M (resp. e"' 1 G T^M) the (1,0) (resp. (0,1)) component 
of the complexification of Cj. 

On the other hand, by Lemma 2.3, for each r + 1 < i < dimG, jii can be 
written, near x, as 



i*i(y) = E%% + °(M 2 )- 

By (2.7) and (2.12), one deduces that 

dim G k k 

E E$ = E«;> 

l=r+ljr' = l j = l 

which, together with (1.10), imply 



(2.12) 



(2.13) 



E |V^)| 2 = E«r 

j=r+l J=l 



(2.14) 



From Lemma 2.3, (2.14), (1.10) and (2.10), one deduces, at x, that 

' dim M dim G 



-i aim ivi aim Lz 

- E E.(^)c(V™(/i^)+4A(e,,V™(/i^))) 

T c (j^)c(^)+i^r 



j=l i=r+l 



dimG / 

+ E -v^TTr [V T(1 '° )M (^^) 

j=r+l V 



+ 



13 



dim G dim G k 

-2y/=i ]T MJv»vj = E v /z Tc(j^)c(^) > -£K|. ( 2 - 15 ) 

i=r+l i=r+l j=l 

From (1.11), (2.9), (2.15) and Lemma 2.3, one gets (2.3). The proof of 
Lemma 2.2 is completed. □ 

Since E is //-positive, using Theorem 1.4, (0.5) and Lemma 2.2, one can 
proceed in the same way as in [TZ1, Sect. 2], with almost no changes, to 
prove Theorem 2.1. That is, we first prove pointwise estimates analogous 
to [TZ1, Prop. 2.2] around each point outside of /i _1 (0), in using Lemma 
2.2 when dealing with critical points of H, and then glue them together to 
get the global estimate (2.1). The essential point in this last gluing step is 
again as in [TZ1] that each Ly t , 1 < % < dimG, vanishes when acting on 
G-invariant sections. We leave the easy details to the interested reader. □ 

Remark 2.4. A notable difference between Lemma 2.2 and [TZ1, Lemma 
2.3] is that even with some of the a/s being negative, one does not have in 
general a strict inequality in (2.3). This means that the /x-positivity of E is 
necessary for Theorem 2.1 (compare with [TZ1, Remark 2.4]). 



b). A proof of Theorem 0.2 



If F is a G-equivariant Hermitian vector bundle over M, we denote by 
Fq its induced bundle on Mq (cf. [TZ1, Sect. 4a)]). 

As in [TZ1], Theorem 2.1 allows us to localize our problem to sufficiently 
small neighborhoods of /i _1 (0). While near /i _1 (0), we can directly apply the 
analysis and results in [TZ1, Sects. 3 and 4a)], which are closely related to 
[BL, Sects. 8, 9], to the G-invariant restriction of D^' ^ T M )® E . 

Combining the above arguments, one deduces using (1.14) the following 
analogue of [TZ1, (4.3)] (of course for different twisted bundles and condi- 
tions), 

dimQ(M, A P '°(T*M) <g> E) G = dimQ(M G , (A P '°(T*M) <g> E) G ). (2.16) 

Now since G is abelian, the normal bundle to /i _1 (0) is equivariantly 
trivial. From this fact one deduces directly the weighted splitting 

(A^°(T*M)) G = ■ A k '°(T* Mq), (2.17) 

k=0 
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where the numbers C^ ir ^ G have been defined in (0.6). 
Theorem 0.2 then follows from (2.16) and (1.17). □ 

Remark 2.5. In view of Remark 2.4, the /z-positivity condition (0.5) can 
not be weakened in general to include the equality as in [TZ1, Theorem 4.2], 
even when /x _1 (0) is nonempty. 

Remark 2.6. Though its proof is of the same method, Theorem 0.2 can 
not be deduced from results in [TZ1] without imposing further conditions. 
The point is that if one wants to apply directly the results in [TZ1] to our 
situation, one needs the condition that at every critical point x G M\// _1 (0) 
of ft, 

dimG 

v -1 2^ ^ r v t > 0, (2.18) 

i=l 

which clearly is not a consequence of (0.5). 
c). A proof of Theorem 0.3 

We now assume that (M,cu,J) is Kahler and G acts on M holomor- 
phically. Furthermore, we assume that E is a G-equivariant holomorphic 
Hermitian vector bundle over M on which G acts holomorphically and that 
V E is the unique holomorphic Hermitian connection. 

The key observation is, similarly as in [TZ1, Remark 1.4], that for any 
T G R we have in this situation 

pAP'°(T*M)®E _ ^/2^ e -TH/2QAP'°(T*M)<8>E e T'H/2 £ TH/2 ^qAP>°(T* M)®Ey £ -TH/2^ 

(2-19) 

Furthermore, one has clearly an analogue of [TZ1, (3.54)]. Thus by the 
same reason as in [TZ1, Sect. 4d)], all the arguments before this subsection 
preserve the Z-grading nature of the twisted Dolbeault complex on M with 
coefficient bundle A P '°(T*M)®E : and this leads to the following holomorphic 
refinement of (2.16) which holds for any integer q > 0, 

h p ' q (E) G - h M -\E) G + ■■■ + {-l) q h p '°{E) G < h°' q ((A p '°(T*M) <g> E) G ) 

-h°' q -\{A p '°{T*M) ® E) G ) + ■■■ + (-l) q h '°{{A p '°{T*M) ® E) G ). (2.20) 
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On the other hand, one also verifies directly that the splitting (2.17) is 
holomorphic in this situation. 

(0.9) is then a consequence of (2.17) and (2.20). □ 



§3. Applications and further extensions 

In this section, we discuss some immediate applications and possible ex- 
tensions of Theorems 0.2, 0.3 as well as the methods and techniques involved 
in their proofs. 

This section is organized as follows. In a), we apply Theorem 0.2 to get 
a vanishing multiplicity result for twisted de Rham-Hodge operators and a 
weighted multiplicity formula for twisted Signature operators. In b), we prove 
a negative analogue of Theorem 0.2, that is, we prove a weighted multiplicity 
formula in the case that '>' is replaced by '<' in (0.5). We also show that 
the strict inequalities are necessary. In c), we discuss briefly the case where 
G g* is not a regular value of the moment map fx. Finally, we discuss in 
d) the applications to the typical example where E is the prequantum line 
bundle over (M,cu). 

a). Applications to twisted de Rham-Hodge and Signature op- 
erators 

Set 

Q dR (M,E)= Q(M,A pfi (T*M)®E)- Q(M, A P '°(T*M) <g> E) 

p=ovcn p=odd 

(3.1) 

and 

dim c M 

Q Sig (M,E)= Q(M,A pfi (T*M)®E). (3.2) 

p=0 

One verifies easily that QdR{M, E) (resp. Qsi^M^E)) is exactly the vir- 
tual vector space associated to the twisted (by E) de Rham-Hodge (resp. 
Signature) operator on M. The following result gives the corresponding 
multiplicity formulas for these operators. 
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Theorem 3.1. Under the same assumptions as in Theorem 0.2, the 
following identities hold, 

dimQ dR (M,£) G = 0, (3.3) 
dimQ Si g(M,£) G = 2 dimG dimQ S ig(M G ,£ G ). (3.4) 

Proof. Theorem 3.1 follows easily from Theorem 0.2 and the definitions 
(3.1), (3.2) with some elementary computation. □ 

Remark 3.2. The assumption that G g* is a regular value of fi is 
essential, particularly for the vanishing property (3.3). This will be discussed 
further in c) and d). 

b). Weighted multiplicity formula for /x-negative bundles 

A G-equivariant Hermitian vector bundle E over (M, a;) with G-invariant 
Hermitian connection V E is said to be /z-negative if at every critical point 
x G M\/x _1 (0) of H = |/i| 2 , one has 

_dimG 

v^T £ ta(x)r^(x) <0, (3.5) 
i=i 

instead of (0.5). 

For any /i-negative bundle E, one can introduce the same deformation 
of twisted Spin c -Dirac operators as in Definition 1.2, but take T — > — oo, 
instead of +oo, to prove the following result. 

Theorem 3.3. If G is abelian and E is ^-negative, then the following 
identity holds for any integer p > 0, 

dim Q(M, A P '°(T*M) <g> E) G 
= (_i)dim G ^ c v-k G . dim Q {MGj A W (rMo) EG y ( 3. 6) 

fc=0 
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Proof. On can proceed similarly as in Section 2 to prove Theorem 3.2. 
The key point to note is that now the analogue of Lemma 2.2 should take 
the following form at any critical point x G M\/i~ 1 (0) of H = |/i| 2 , 



dimM 



dimG 



E (c(e J )c(V™X«)+4A(e,,VW 



) ^Tr 


v r(1 '"' M x w 


' 2 





aim dim M 

+ o E (v /r Tc(J\/ J )c(\/ l ) + |\/,| 2 -4 v / ^TA(JV;,\/ i )) < E Kl- ( 3 - 7 ) 

1 i=l j=l 

We leave the details to the interested reader. □ 



As immediate applications, one gets analogues of Theorem 3.1 for /i- 
negative bundles. One also gets Morse type inequalities as refinements of 
(3.6) in holomorphic situations. 

Remark 3.4. We use a simple example to illustrate that even with 
A t_1 (0) 7^ 0) one can n °t weaken the /i-positive (resp. /i- negative) assumption 
in (0.5) (resp. (3.5)) by an equality to get a weighted multiplicity formula 
similar to what happens in [TZ1, Theorem 4.2]: Taking M to be P 1 with its 
standard S* 1 action and E = C, one verifies directly that dimQdR(^, C) G = 
dimQ dR (M,C) = 2^0. 

Remark 3.5. It is also clear that the abelian condition on G is essential 
for our argument. In fact, an example due to Vergne (cf. [JK, pp.686]) 
shows that the abelian condition on G is necessary for Theorem 3.3 in the 
case where p = and E is the dual of the prequantum line bundle over 
(M,lo) (when the latter exists). 

c) . The case where G g* is a singular value of /i 

We now make a brief discussion on the possible generalizations of our 
main results to the case where G g* is a singular value of /i. 

When p = and E is the prequantum line bundle over (M, u) (when it 
exists), the quantization formula for singular reductions has been established 
by Meinrenken-Sjamaar [MS] (see also [TZ3] for an analytic treatment in 
certain situations). One notable feature in this case is the phenomenon that 
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'the singular value is removable' in the perturbative singular quantization 
formula [MS, Theorems 2.5, 2.9] (see also [TZ3, Theorem 0.1]). However, as 
we will see, the situation for the case where p is nonzero is rather different. 

In the simplest case where G is the circle, a fairly general singular localiza- 
tion formula, which can indeed be applied to our situation, has been proved 
in [TZ2, Theorem 6.7]. To be more precise, let V be the Killing vector field on 
M generated by the unit base of g and let F Q = /i _1 (0) fl {x e M; V(x) = 0} 
be the subset of the fixed points of the G action contained in /i _1 (0). Then 
using the same notation as in [TZ2, Sect. 6c) and Appendix], one has the 
following result, which can be proved by a combination of the arguments in 
Section 2 with those in the proof of [TZ2, Theorem 6.7]. 

Theorem 3.6. If G = S 1 , E is ji-positive and G g* is a singular value 
of \i, then 

dim Q(M, A P '°(T*M) E) G 
= E CLf G • dim Q(M Gfi -, A fc >°(T*M Gi0 -) ® E Gfi -) + indD^ M ^ E (V) 

k=0 

= E C p d ^ G ■ dimQ(M G , 0+ , A fc '°(T*M G , 0+ ) ® E Gfi+ ) + mdD^ T * M) ® E (-V). 

k=0 

(3.8) 

Corollary 3.7. Under the same conditions as in Theorem 3.6, one has 
dim QdR (M,Ef = ind<7°™ 8 >) - ind/jC°™>) 

= ind<7° (T * M) n-Vl - ind<;;'° (T * M) ^(-\/). (3.9) 

Remark 3.8. In the next subsection, we will show that even when E is 
the prequantum line bundle over (M, a;), the contributions of F to the right 
hand sides of (3.8), (3.9) may well be nonzero. This explains the essential 
difference between the p = and p ^ cases as mentioned above. 

Remark 3.9. If G is of higher rank, one can apply Theorems 3.6, 3.7 
inductively to get localization formulas for dimQ(M, A P,0 (T*M) <g> E) G and 
dimQdR(-^, E) G respectively. 
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d). The case where E is the prequantum line bundle over (M,u) 



In this section, we assume E is the prequantum line bundle L over (M, u), 
of which we assume the existence. Then L is a G-equivariant Hermitian 
vector bundle over M with G-invariant Hermitian connection V L such that 

^(V*)"=„. (3,0) 

Furthermore, we assume that /i verifies the Kostant formula [Ko] (cf. [TZ1, 
(1.13)]) 

L v s = V v s - 27rv /Z T(/i, V)s, s G T{L), V G g. (3.11) 
The following result is clear. 



Proposition 3.10. L is ^.-positive. 



One of the novelties for the prequantum line bundle is that there is a 
standard shifting trick to reduce the computation of dimensions of nontrivial 
components of the G- representation Q(M, A P '°(T*M) <g> L) to those of trivial 
components of G-representations of the form Q(M, A P '°(T*M) <g> E) with E 
to be '/x-positive'. 

To be more precise, for any (,GZ, 1 < i < dimG, set 

£ = Clh + • ■■UmGhdunG & g* ■ (3.12) 

Let be the G-equivariant trivial complex line bundle over M with G- 
invariant Hermitian connection V c? , on which G acts by 

L v s = Vy ( s + 27rv /z T(£, V)s, s G T (L) , V eg. (3.13) 

The existence of is clear. 
Set 

L ? = L®C 5 . (3.14) 

Then is canonically a G-equivariant Hermitian vector bundle over M with 
the tensor product connection V L « . Furthermore, G acts on through the 
formula 

L v s = Vf f s - 2nV^l(fi - £, V)s, s G T(L ( ), V G g. (3.15) 
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Since G is abelian, /ig := // — £ may also be regarded as a moment map for 
the Hamiltonian action of G on (M, a;). One then has the following extension 
of Proposition 3.10, which can be verified directly. 

Proposition 3.11. is ^-positive. 

Now let Q(M, A P '°(T*M) <g> L) ( denote the £-eigen-component of the G- 
representation Q(M, A p '°(T*M)(g)L). That is, L v acts on Q(M, A P '°(T*M)® 
L)^ by multiplication by 2rcy/—l(£, V). Then one verifies directly that 

dim Q(M, A P '°(T*M) ® L)^ = dim Q(M, A P '°(T*M) <g> L C ) G . (3.16) 

This is the shifting trick mentioned above. 

Now when £ is a regular value of /i, in view of Proposition 3.11 one 
can apply Theorem 0.2 to get a weighted multiplicity formula calculating 
dim Q(M, A P >°(T*M)®L 5 ) G and thus the f -multiplicity of Q(M, A P -°(T*M)® 
L) in terms of quantities on the /i^-symplectic reduction M G ^ = /i _1 (£)/G. 
While when £ is not a regular value of //, one may first apply Theorem 3.6 
and then an induction procedure to calculate dimQ(M, A P '°(T*M) <S>L)^ via 
(3-16). ^ 

By summing over all these £'s, one has clearly that 

dimQ(M, A P '°(T*M) ® L) = ]T dim Q(M, A P '°(T*M) <g> L) € . (3.17) 

In particular, in the case where G — S 1 , if for any such £ denote by = 
A* _1 (0 l" 1 ^ where F is the fixed point set of the G action on M, then one 
can combine the above reasoning with Corollary 3.7 to get 

dim Q dR (M,L) = £ (ind<;'° (T * M) ^(V) - ind D^ M ^(V)) . 

(3.18) 

Remark 3.12. Note that if £ is not contained in the image of the /x, then 
it is automatically a regular value of /i. In this case, M G ^ = 0. By Theorem 
0.2, one sees immediately that in the summations in (3.17) and (3.18), the £ 
actually runs through the integral lattice points contained in the image of \x. 
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Now if the Euler characteristic e(M) of M is nonzero, then from (3.18) 
and the Atiyah-Singer index theorem [AS], which gives that 

dim Q dR (M, L) = e(M), (3.19) 

one deduces by (3.17) that at least one of the terms dimQ(M, A even '°(T*M)<g> 
L) s and dimQ(M, A odd '°(T*M) <g> L) ? should be nonzero. In view of (3.16), 
(3.18), Theorem 3.6 and Corollary 3.7, this provides a concrete example 
mentioned in Remark 3.8. 
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